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(54) Display control apparatus and display system 



(57) A display system for receiving image data from 
a plurality of image sources and displaying the received 
image data on a display unit. The display system in- 
cludes: an input unit for inputting a control signal sup- 
pliedfrom an input/output unit, a controller for controlling 
an image to be displayed on the display unit in accord- 



ance with the control signal input from the input unit; an 
image selection unit for selecting a desired image from 
images displayed on the display unit; and a . switching 
unit for switching an image to be controlled by the con- 
troller in accordance with the image selected by the im- 
age selection unit. 
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[0011] Fig. 2 is a diagram showing examples of a dis- 
play layout of a display device when a plurality of image 
sources are input. 

[0012] Fig. 3 is a diagram showing the structure of a 
selection portion. 

[001 3] Figs. 4A, 4B and 4C are diagrams showing ex- 
amples of images and a display pointer on a display de- 
vice. 

[0014] Fig. 5 is a block diagram showing the structure 
of a display system according to a second embodiment 
of the invention. 

[001 5] Fig. 6 is a block diagram showing the structure 
of a display system according to a third embodiment of 
the invention. 

[0016] Figs. 7 A and 7B show a display screen, a win- 
dow, a main pointer, and an image source pointer on a 
display control system, Fig. 7A illustrates a control by 
the image source pointer, and Fig. 7B illustrates a con- 
trol by the main pointer. 

[0017] Figs. 8A and 8B show a display screen, win- 
dows, a main pointer, and an image source pointer on 
a display control system, Fig. 8A illustrates a selection 
of a window of an image source 1-1 with the main point- 
er, and Fig. 8B illustrates a selection of a window of an 
image source 1 -2 with the main pointer. 
[0018] Fig. 9 is a diagram illustrating a relation be- 
tween a general software configuration and an embod- 
iment software configuration under the graphic user in- 
terface (GUI) environment on the image source side. 
[0019] Fig. 10 is a diagram showing an operation 
screen of an application running under the GUI environ- 
ment on the image source side according to an embod- 
iment. 

[0020] Figs. 1 1 A and 1 1 B show the display screen of 
a display control system, Fig. 1 1 A illustrates a selection 
of a window of an image source 1 -1 with a main pointer, 
and Fig. 11 B illustrates a selection of a clip board icon 
Y1 with a main pointer. 

[0021] Figs. 12Aand 12B show the display screen of 
a display control system, Fig. 12A illustrates a selection 
of a window of an image source 1 -1 with a main pointer, 
and Fig. 12B illustrates a selection of a window of an 
image source 1-2 with the main pointer. 
[0022] Fig. 1 3 is a flow chart illustrating the operation 
of a main process to be executed when data is input 
from an infrared data control portion of a control portion 
6 of a display control system 40. 

[0023] Fig. 14 is a flow chart illustrating a process R1 
in the flow chart shown in Fig. 1 3. 
[0024] Fig. 15 is a flow chart illustrating a process R2 
in the flow chart shown in Fig. 13. 

[0025] Fig. 16 is a flow chart illustrating a process R3 
in the flow chart shown in Fig. 13. 

[0026] Fig. 17 is a flow chart illustrating a process R4 

in the flow chart shown in Fig. 1 3. 

[0027] Fig. 18 is a flow chart illustrating a process R5 

in the flow chart shown in Fig. 13. 

[0028] Fig. 19 is a flow chart illustrating the operation 



of a main process to be executed when data is input 
from a packet control portion of the control portion 6 of 
the display control system 40. 

[0029] Fig. 20 is a flow chart illustrating a process P1 
s in the flow chart shown in Fig. 19. 

[0030] Fig. 21 is a flow chart illustrating a process P2 

in the flow chart shown in Fig. 19. 

[0031] Fig. 22 is a flow chart illustrating a process P3 

in the flow chart shown in Fig. 19. 
10 [0032] Fig. 23 is a flow chart illustrating a process P4 

in the flow chart shown in Fig. 19. 

[0033] Fig. 24 is a flow chart illustrating a process P5 

in the flow chart shown in Fig. 19. 

[0034] Fig. 25 is a flow chart illustrating a process P6 
15 in the flow chart shown in Fig. 19. 

[0035] Fig. 26 is a flow chart illustrating a process P7 

in the flow chart shown in Fig. 19. 

[0036] Fig. 27 is a flow chart illustrating the operation 

of a process to be executed when a message is input 
20 from GUI of the control portion 6 of the display control 

system 40. 

[0037] Fig. 28 is a flow chart illustrating a process A1 

in the flow chart shown in Fig. 27. 

[0038] Fig. 29 is a flow chart illustrating a process A2 

25 in the flow chart shown in Fig. 27. 

[0039] Fig. 30 is a flow chart illustrating a process A3 
in the flow chart shown in Fig. 27. 
[0040] Fig. 31 is a flow chart illustrating a process A4 
in the flow chart shown in Fig. 27. 

30 [0041] Fig. 32 is a flow chart illustrating the operation 
of a process to be executed when a display software 
loading command is obtained from a serial communica- 
tions driver of the control portion 6 of the display control 
system 40. 

35 

DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

(First Embodiment) 

40 

[0042] Fig. 1 is a block diagram showing the structure 
of a display system according to the first embodiment of 
the invention. 



[0043] Each block shown in Fig. 1 will be described. 
In this embodiment, information supplied from four in- 
dependent image sources (such as note PC) is dis- 

50 played on one monitor and an input/output device of 
each image source, such as a mouse, a keyboard, a 
remote controller and a speaker, is controlled in coop- 
eration with image data displayed on the monitor. The 
.number of image sources may be larger than or smaller 

55 than four. 

[0044] Reference numeral 31 represents a display 
system. Reference numerals 1-1, 1-2, 1-3 and 1-4 rep- 
resent sources of image signals (hereinafter called *im- 



45 [Description of Each Component] 
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plied from each image source 1. Reference numeral 27 
represents a D/A converter for converting input digital 
sound data into analog data. Reference numeral 28 rep- 
resents a sound amplifier, and reference numeral 29 
represents a speaker of the display system 31 . 
[0050] The operation of the display system of the em- 
bodiment will be described with reference to Fig. 1 . 

[Initialization] 

[0051] Each image source 1 and display system 31 
communicate with each other via a bi-directional serial 
communications line of the input portion 2 after powers 
are turned on. Each input portion 2 supplies the image 
source 1 with data such as the numbers of dots, display 
lines, display colors, and a video output timing. The 
communications are performed by using a communica- 
tions protocol predetermined both by the input portion 2 
and image source 1 . For example, a DDC (Display Data 
Channel 1) of VESA (Video Electronics Standards As- 
sociation) in USA may be used, and EDID (Extended 
Display Identification Data) of VESA may be used as the 
data format. In the communications, the numbers of dis- 
play dots, display lines and display colors of the display 
device 13 are supplied, or the display formal predeter- 
mined at the control portion 6 may be supplied. In ac- 
cordance with the received information, each image 
source 1 outputs image data and its control signals to 
the input portion 2. 

[0052] The input portion 2 always monitors a connec- 
tion identification signal to be supplied from the image 
source in order to check the number of image sources 

1 connected to the display system 31 . The input portion 

2 receives the connection identification signal from the 
image source as a logical binary signal "1" or "0". 
[0053] If the connection cable is cut or the power of 
the image source is turned off, the logical signal "O" is 
supplied because the input portion 2 is terminated with 
a resistor. The display system 31 can therefore recog- 
nize that image data is not input. This monitor informa- 
tion is supplied to the control portion 6 at a predeter- 
mined interval. 

[0054] The control portion 6 first detects the connec- 
tion identification signal of the input portion 2-1. If the 
connection identification signal is "1", the input portion 
2-1 sends the received image data to the display format 
conversion portion 3-1 to covert the format of the image 
data, whereas if the connection identification signal is 
"0", the control portion 6 checks the connection identifi- 
cation signal of the next input portion 2-2 and scans the 
input portion 2 until the connection identification signal 
of "1 " is detected. If all the connection identification sig- 
nals are "0", the display system 31 enters a power save 
mode to lower a power supply to the constituent ele- 
ments excepting the control portion 6, selection portion 
14, packet distribution control portion 1 5, packet control 
portion 16 with FIFO, FIFO memory 17, infrared data 
conversion portion 18, infrared data control portion 19, 



and infrared data receiving portion 20. 
[0055] Display data in ROM of the control portion 6 is 
stored in the superposition data memory 10 via the bus 
and data controller 8. 
s [0056] Instead of ROM of the control portion, the dis- 
play data may be supplied to the control portion 6 via 
the selection portion 14 and packet control portion 16 
with FIFO to be described later. 

10 [Connection of One Image Source] 

[0057] The case wherein the connection identification 
signal of only the input portion 2-1 is "1 M and the signals 
of the other input portions are "0 W , will be described. 

75 [0058] The input portion 2-1 outputs image signals 
having a desired format to the display format conversion 
portion 3-1 . If the supplied display format is equal to the 
display format (dot number, line number and color 
number) of the display device 13, the supplied image 

20 data is passed to the bus interface, whereas if they are 
different, the supplied display format is converted into 
the display format of the display device 1 3 by processing 
the supplied image data to have proper dot number and 
line number. If the number of colors of the supplied im- 

25 age data is larger than the number of colors the display 
device 1 3 can display, the number of bits of image data 
is reduced at an intermediate image data process such 
as a dither process. 

[0059] The image data output from the display format 
30 conversion portion 3-1 is stored in the frame memory 9 
via the bus interface 4-1, bus controller 5 and frame 
memory controller 7. 

[0060] The frame memory 9 has a plurality of hierar- 
chical levels. The level at which the input data is stored 
35 js controlled by the bus controller 5 under the control of 
the control portion 6. 

[0061] Image data stored in the frame memory 9 is 
output to the bus at a timing controlled by the control 
portion 6 and to the output display format conversion 
40 portion 11. In this case, the hierarchical level of image 
data to be output is determined by the bus controller 5 
under the control of the control portion 6. 
[0062] In addition to the input image data, display data 
stored in the superposition data memory 10 is output to 
45 the bus via the superposition data controller 8 under the 
control of the control portion 6 to be displayed on the 
display device 13. Either the superposition data or the 
input image data output to the bus via the frame memory 
controller 7 is selected by the output display format con- 
so version portion 11 and the selected data is supplied to 
the display driving controller 12. This selection timing is 
determined by the control portion 6. 
[0063] The output display format conversion portion 
11 converts the selected image data into data having a 
5S data bus width and the like suitable for the display driv- 
ing controller 1 2. The image data converted by the con- 
version portion 11 is output to the display driving con- 
troller 12 which generates a drive signal for driving the 
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priority order of the hierarchical level at which image da- 
ta is stored. 

[0078] The display example (a) shown in Fig. 2 is re- 
alized in the following manner. The display format con- 
version portion 3-1 stores the image data of the input 5 
portion 2-2 in the frame memory 9 at the hierarchical 
level of the second priority order without performing 
scaling. The display format control portion 3-2 performs 
scaling of the image data of the image source 1 -2 to re- 
duce the image data by thinning input display dots and io 
lines. The reduced image data is stored in the frame 
memory 9 at the hierarchical level of the first priority or- 
der 

[0079] In the display example (b), both the display for- 
mat conversion portions 3-1 and 3-2 perform scaling to is 
display two images without superposing them. 
[0080] In the display examples (c) and (d), the images 
(1-1) and (1-2) of the display examples (a) and (b) are 
exchanged. In the display example (e), both the display 
format conversion portions 3-1 and 3-2 do not perform 20 
scaling and the image data is stored in the frame mem- 
ory 9 at the hierarchical level of the first priority order 
after each image data of the images (1-1) and (1-2) is 
processed at least in the unit of one pixel by a calculation 
circuit of the frame memory controller 7. 
[0081] As described above, when two or more images 
are input, the display layout of the images can be freely 
changed by using scaling at the display format conver- 
sion portions 3 and the hierarchical level of the frame 
memory 9. 

[0082] This display layout change can be performed 
by instructing the control portion 6 to present one of the 
display examples (a) to (e) shown in Fig. 2 by pressing 
the button 30 of the display pointer controller 21 and no- 
tifying the control portion 6 of the depression via the re- 
ceiving portion 20 and infrared data control portion 19. 
Each time a display layout change is instructed, the con- 
trol portion 6 notifies a change in the display parameters 
to the display format conversion portion 3, bus controller 
5 and frame memory controller 7. 

[Switching between Input/Output Devices by Using 
Display Pointer] 

[0083] Next, a method of switching between input/out- 
put devices of image sources by using a display pointer 
will be described which method is a characteristic fea- 
ture of the invention. 

[0084] As described earlier, the display pointer con- 
troller 21 has the joy stick (mouse) 22 as an input device 
of image sources, the control button 23 functioning as 
the confirmation button of the joy stick 22, the joy stick 
24 for controlling a pointer displayed on the display de- 
vice - 1 3 and superposed upon the input image data, and 
the control button 25 functioning as the confirmation but- 
ton of the joy stick 24. In this embodiment, the joy stick 
is used as the input device of the image source 1 , an 
input device such as a keyboard and a tablet may also 



be used and the output device such as a liquid crystal 
display and a small printer may also be used. 
[0085] After an initialization process of determining a 
reference point of the X and Y coordinates is performed 
while viewing a superposition pointer displayed on the 
display device 1 3, the X and Y coordinate values of the 
joy stick 22 are supplied to the control portion 6 via the 
receiving portion 20 and infrared data control portion 1 9. 
The control portion 6 calculates coordinate values of 
four corner points of a window of the display device in 
which an input image is currently displayed, by using the 
numbers of dots and lines used by each display format 
conversion portion 3, and always stores the calculated 
coordinate values. 

[0086] The X and Y coordinate values sent from the 
joy stick 24 are sent to the control portion at a predeter- 
mined interval. In order to display the pointer figure 
stored in the superposition data memory 10 at the posi- 
tion indicated by the coordinate values, the superposi- 
tion data is output to the output display format conver- 
sion portion 1 1 . At this time, the joy stick 22 which is one 
of the image source input/output devices is connected 
to the image source 1 -1 via the receiving portion 20, in- 
frared data control 'portion 19, infrared data conversion 
portion 18, FIFO memory 17, packet distribution control 
portion 1 5 and selection portion 1 4. 
[0087] The pointer figure in the superposition data 
memory 10 may be changed by each image source via 
the route of the selection portion 1 4, packet control por- 
tion 16 with FIFO, control portion 6, bus controller 5 and 
superposition data controller 8, or may be replaced by 
a pointer figure stored in ROM of the control portion 6 
via the route of the bus controller 5 and superposition 
data controller 8. 

[0088] If a user wants to use the joy stick 22 as the 
input/output device of the image source 1-2 instead of 
the image source 1 -1 , the user moves the superposition 
data figure to the area where the image (1-2) is dis- 
played by using the joy stick 24 and depresses the co- 
ordinate confirmation control button 25. Information in- 
dicating a depression of the control button 25 is trans- 
ferred to the control portion 6 via -the receiving portion 
20 and infrared data control portion 19. Through calcu- 
lation of the X and Y coordinate information when the 
button depression information is received and the al- 
ready known coordinate values of four corner points of 
the image data display window of each image source, 
the control portion 6 judges the display window in which 
the X and Y coordinate values of the display pointer joy 
stick 24 are located to thereby select the desired image 
source. 

[0089] The control portion 6 supplies a control signal 
to disconnect once the selection portion 1 4, and after a 
predetermined time lapse, the serial data line of the im- 
age source 1-2 is connected to the packet distribution 
control portion 15. 

[0090] With the above operations, peripheral devices 
using a serial data protocol capable of a hot plug-in con- 



30 



35 



40 



45 



SO 



7 



BNSDOCID: <EP 09 50944 A2_l_> 



15 



EP.O 950 944 A2 



16 



input, the display of the image data can be changed free- 
ly in accordance with the scaling by the display format 
conversion portion 3 and the hierarchical level of the 
frame memory 9. Therefore, the display control of imag- 
es entered from a plurality of image sources, such as a 
change in the display layout, can be performed easily 
by using the display controller 21 which is the input/out- 
put device common to all the image sources. 

(Second Embodiment) 

[01 07] Fig. 5 is a block diagram showing the structure 
of a display system according to the second embodi- 
ment of the invention. 

[0108] In the second embodiment, four television tun- 
ers are used as the image sources. The other structures 
are similar to the first embodiment. 
[01 09] Television signals received by an antenna 209 
are detected and frequency-modulated by television 
tuners 205 to 208. Each encoder 201 to 204 decodes 
television signals having a standard signal format such 
as NTSC, PALandSECAM into composite video signals 
and sound signals. 

[01 1 0] The decoded composite signal is input to each 
input portion 2 and displayed on the display device in 
the manner similar to the first embodiment. 
[0111] For the control of each tuner such as a selec- 
tion of a broadcast station, on/off information of the but- 
ton of the display pointer controller 21 is supplied to the 
selection portion via the serial control signal line and via 
the route of the receiving portion 20, infrared data con- 
trol portion 19, infrared data conversion portion 18, FIFO 
memory 17 and packet distribution control portion 15. 
The control portion 6 controls the selection portion 14 to 
select one of the tuners 205 to 208 by the joy stick 24 
which tuner receives television data of a display image. 
In this manner, the television program of the display im- 
age received by the tuner and selected by the joy stick 
24 can be changed easily. In response to a change in 
the television program, sound data is reproduced from 
the speaker 29 via the packet distribution control portion 
15, packet control portion 26, D/A convenor 27 and 
sound amplifier 28. 

[0112] In the second embodiment, the control of the 
image entered from the television tuner, such as a 
change in the television program, can be performed with 
advantageous effects similar to the first embodiment. 
[0113] In the first and second embodiments, control 
programs may be supplied from an external apparatus, 
and in accordance with the supplied control programs, 
the display of an image on the display device 1 3 may 
be controlled. In this case, the control is executed by the 
control portion 6 in place of the display pointer controller 
21. In the second embodiment in particular, if a digital 
television broadcast station superposes control data, 
such as display image scaling and superposition, upon 
television image data, various controls such as display 
control can be performed by using the superposed con- 
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trol data, without controlling the control portion 6 by us- 
ing the display pointer controller 21. In this case, the 
control data is supplied to the control portion 6 via the 
packet control portion 16 with FIFO without using the 

s relays 101 to 104 of the selection portion 14. 

[0114] In the first and second embodiments, when da- 
ta of the display pointer and joy stick is transferred from 
the display pointer controller 21 to the display system 
31 , infrared light is used. The invention is not limited only 

io to infrared light, but radio waves or signalling lines may 
also be used. 

(Third Embodiment) 

15 [011 5] Fig. 6 is a block diagram showing the structure 
of a display system according to the third embodiment 
of the invention. In the third embodiment, information 
supplied from four independent image sources is dis- 
played on one monitor. The number of image sources 

20 may be larger than or smaller than four. In Fig. 6, com- 
ponents having similar structures to those shown in Fig. 
1 are represented by using identical reference numer- 
als, and the description thereof is omitted. 
[0116] Referring to Fig. 6, image sources 1-1, 1-2, 1-3 

2S and 1 -4 have user information input means (not shown) 
such as a mouse for entering user information indicat- 
ing, for example, a position of an image source pointer. 
Reference numerals 14-1, 14-2, 14-3 and 14-4 repre- 
sent a packet control portion with FIFO which is used 

30 for controlling a timing of transmitting and receiving high 
speed serial communications packet data between the 
image sources 1 -1 to 1 -4 and the control portion 6 of the 
display system 40 of this embodiment. In this embodi- 
ment, by using these high speed bi-directional serial 

35 communications packet control portions 14-1 to 14-4, 
switching between update and nop- update of windows 
for the image sources ; controlling a pointer and other 
controls are performed. 

[0117] Examples of multiplexed packet data transfer 
40 are IEEE 1394, USB (Universal Serial Bus) or the like. 
USB is most suitable for this embodiment because pres- 
ently available desk top type PC's and note type PC's 
are provided with USB ports and USB supports a hot 
plug-in connection (allowing plug-in and plug-out during 
45 operation). From the viewpoint of the characteristic fea- 
tures of the invention, serial communications of other 
types are also applicable with similar advantageous ef- 
fects. Apparatuses for USB include a USB host and USB 
devices. Generally, the USB host is a personal compu- 
50 ter, and the USB devices are peripheral apparatuses 
such as a mouse, a keyboard and a printer. In this em- 
bodiment, the packet control portions 14-1 to 14-4 func- 
tion as the'USB devices. 

[0118] The operation of the display system 40 of the 
55 third embodiment will be described with reference to Fig. 
6. 
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the main pointer 25. The superposition data memory 10 
has a capacity corresponding to the screen size of the 
display device and has a depth of at least one bit. 
[0130] The above operation state is illustrated in Fig. 
7B. Since it is assumed in this description that only one s 
image source is connected, only the image of the image 
source 1 -1 is displayed on the screen of the display de- 
vice. Therefore, in Fig. 7B, the on — » off operation of the 
main button 25 selects the whole image of the image 
source 1-1 . 10 
[0131] The above operations of the control portion 6 
are illustrated in the process R0 (r0-1 ) in Fig. 1 3 and the 
process R1 (r1-1 to r1-5) in Fig. 14. 
[01 32] The control portion 6 instructs the bus control- 
ler 5 to enable to control to output or not to output the 15 
data in the superposition data memory 10 to the output 
display format conversion portion 11 in order to display 
or not to display the main pointer. This control is per- 
formed by depressing the button 31 of the remote con- 
troller. Similar to the above description, the ON/OFF in- 20 
formation of the button 31 is once converted into pack- 
eted infrared data and supplied to the control portion 
(process RO (rO-4) in Fig. 3 and process R4 in Fig. 17). 

[Connection of Two or More Image Sources] 25 

[0133] If two or more image sources are connected, 
the processes similar to the first embodiment are exe- 
cuted. 

30 

[Switching between Input Device Controls of Image 
Source by Main Pointer] 

[0134] Next, a method of switching between input de- 
vice controls of an image source by the main pointer will 35 
be described. 

[0135] The remote controller 21 has the subsidiary joy 
stick 22 and subsidiary button 23 which are input devic- 
es of the image source and the main joy stick 24 and 
main button 25 for controlling the main pointer displayed 40 
on the display device and superposed on the input im- 
age data. 

[0136] The control portion 6 of the display system 40 
calculates the four corner coordinate values of the win- 
dow currently displaying an image, from the numbers of 45 
dots and lines to be used for scaling by the display for- 
mat conversion portions 3-1 to 3-4, and always stores 
the calculated coordinate values. 

[0137] The X and Y coordinate values calculated by 
the control portion 6 in accordance with the X and Y co- 50 
ordinate motion amount supplied from the main joy stick 
24 are used for drawing a pointer figure of a mouse or 
the like in the superposition data memory 10 at the lo- 
cation corresponding to the X and Y coordinate values 
to thereafter output the superposition data to the output 55 
display format conversion portion 11. In this case, the 
subsidiary joy stick 22 functioning as one of the input 
devices of the image source 1-1 by the operations de- 



scribed previously is being connected to the image 
source 1-1 via the infrared data.receiving portion 20, in- 
frared data control portion 19. control portion 6, and 
packet control portion 14-1 , to thereby transfer a packet 
of a X and Y coordinate motion amount and the like and 
allow the mouse of the image source 1-1 to move in ac- 
cordance with the operation of the subsidiary joy stick 
23 (Fig. 8A). 

[0138] If a user acting upon the remote controller 21 
wishes to use the subsidiary joy stick 22 as the input 
device for controlling the image source 1-2 instead of 
the image source 1 -1 , the user moves the main pointer 
to the area in which the image data 1-2 is displayed by 
using the main joy stick 24, and then depresses the main 
button 25. Information of a depression of the button 25 
is transferred to the control portion 6 via the infrared data 
control portion 19. By using the X and Y coordinate in- 
formation when the button information is received and 
the already known four comer coordinate values of the 
image data display window of each image source, the 
control portion 6 checks through calculation the window 
having the X and Y coordinate values of the main pointer 
(process R0 (r0-1) in Fig. 13 and process R1 (r1-1 to 
M-4) in Fig. 14). 

[0139] If the time while the main button 25 is de- . 
pressed (button ON) is a predetermined time or shorter 
and if it is recognized that the current coordinate position 
is on a window different from a window when the main 
button 25 was depressed previously, then the control 
portion 6 switches the packet destination of USB serial 
communications. Namely, data write into the packet 
control portion 14-1 in order to supply the image source 
with the X and Y coordinate motion information of the 
subsidiary joy stick 22 and the ON/OFF information of 
the subsidiary button 23, is stopped, and data write into 
the packet control portion 14-2 starts for the image 
source 1-2. The color of the frame of the selected win- 
dow is changed to explicitly shown the currently select- 
ed widow, and the color of the frame of the previously 
selected window is changed to a usual color (process 
R1 (rl -6, r1 -9 to r1 -1 1 ) in Fig. 1 4). 
[01 40] Therefore, the user can control a mouse cursor 
or the like displayed on the window of the image source 
1-2 by using the subsidiary joy stick 22 and subsidiary 
button 23 as if the subsidiary joy stick 22 and subsidiary 
button 23 are pointing devices of the image source 1 -2. 
[01 41] In this embodiment, USB is used for serial data 
communications. In most of serial communications such 
as USB, communications data such as a command is 
periodically or non-periodically sent from a host (in the 
embodiment, image source) to even an input device 
such as a pointing device which has generally a data 
flow only from the device to the host. If the input device 
does not respond to the command, some problem oc- 
curs. An example of such a command is a reallocation 
command of device addresses on the bus. If such a 
command is not properly responded, the host (image 
source) sometimes recognizes that the device is in an 
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ing data and more particularly a partial area of the mem- 
ory. This memory area is hereinafter called a "clip 
board". Y1 in Figs. 11 A and 118 represents an icon 
which is written in the superposition data memory by the 
control portion and is called a "common clip board icon". 
It is herein assumed that two image sources 1 -1 and 1 -2 
are connected. 

[01 54] First, a user moves the main pointer to the im- 
age of the image source 1 -1 by using the joy stick 24 of 
the remote controller and depresses (button ON) the 
button 25 to select the image of the image source 1-1 . 
If the time while the button is depressed is a predeter- 
mined time (e.g., 1 second) or longer, the control portion 
6 controls the bus interface 4-1 to stop the update of the 
image of the image source 1 -1 (process R1 (r1 -1 to M -8) 
shown in Fig. 14). This operation state is illustrated in 
Fig. 11 A. The control portion 6 therefore enters a "drag 
state". It the user moves the main pointer to the "com- 
mon clip board icon" Y1 while depressing the button 25 
and releases the depression (button OFF) of the button 
25 (Fig. 11 B), then the control portion 6 reads the win- 
dow area corresponding to the image of the image 
source 1 -1 from the frame memory and stores the read 
data in RAM of the control portion 6. Thereafter, the con- 
trol portion 6 instructs the bus interface 4-1 to start again 
the display update (process R1 (r1-2) shown in Fig. 14 
and process R5 (r5-1 to r5-3, r5-5 to r5-7) shown in Fig. 
18). 

[01 55] With the above operations, the image of the im- 
age source 1 -1 is stored in the RAM area of the control 
portion, i.e., in the "common clip board". Thereafter, the 
control portion writes data into the superposition data 
memory so as to change the color or shape of the "com- 
mon clip board icon" Y1 to explicitly show that the data 
is being stored in the "common clip board". 
[01 56] If another user operating the image source 1 -2 
wishes to copy the data (in this case, image of the image 
source 1-1 ) in the common clip board to its own image 
source 1-1 side, a selection switch "common — > local" 
w3 ol the application shown in Fig. 10 is selected and a 
copy button is depressed so that a command is trans- 
mitted to the control portion 6 over USB serial commu- 
nications to urge the control portion 6 to transmit the da- 
ta in the common clip board (process AO (a0-3) shown 
in Fig. 27 and process A3 (a3-1 , a3-3) shown in Fig. 30). 
Upon reception of this command, the control portion 6 
divides the data stored in the internal RAM into data 
pieces and write them into the packet control portion 
14-2 in order to transmit the data over USB serial com- 
munications (process P0 (p0-3) shown in Fig. 19, proc- 
ess P2 (p2-4) shown in Fig. 21, and process P6 (p6-1, 
p6-3) shown in Fig. 25). The application of the image 
source 1 -2 reconfigures the divided data pieces trans- 
mitted over USB serial communications, and transfers 
the reconfigured data to a "local clip board" under man- 
agement of the GUI environment running on the image 
source (process A3 (a3-4) shown in Fig. 30). The mem- 
ory area for temporarily saving data such as "clip board" 



of this embodiment is generally defined in the GUI en- 
vironment. The "local clip board" means such a memory 
area. The area in which data is actually stored may be 
a memory IC or a memory disk medium such as a hard 

5 disk and a DVD. 

[0157] With the above operations, it is possible to 
transfer the image data displayed on the display system 
to another image source, by using the saving function 
of the "common clip board". 

w [PC PC] 

[0158] It is possible for the image sources to share 
image data by using the "common clip board". 
[0159] For example, a user operating the image 
source 1-1 transfers data to be shared to the local clip 

is board under management of the GUI environment, and 
thereafter selects the selection switch w3 "local -> com- 
mon" of the application shown in Fig. 10 and depresses 
the copy button. In this case, the application LI first 
sends a command to the control portion 6 over USB se- 

20 rial communications to notify that the data is transferred. 
The application then reads the data from the local clip 
board under management of the GUI environment, con- 
verts the data into a packet and transfers the packet to 
the control portion 6 over USB serial communications 

25 (process AO (aO-3) shown in Fig. 27 and process A3 
(a3-2, a3-5, a3-6) shown in Fig. 30). 
[0160] In response to the command, the control por- 
tion 6 receives the data via the packet control portion 
14-1, saves the data in the internal RAM, and changes 

30 the color or shape of the "common clip board icon" of 
the superposition data memory 10 to explicitly show that 
the data is being saved in the "common clip board" 
(process P0 (pO-3) shown in Fig. 19, process P2 (p2-4) 
shown in Fig. 21 , and process P6 (p6-2, p6-4) shown in 

35 Fig. 25. 

[0161] If another user operating the image source 1 -2 
wishes to copy the data saved in the "common clip 
board" to the image source 1-2 side, the user selects 
the selection button w3 "common -> local" of the appli- 
40 cation shown in Fig. 10 and depresses the copy button 
so that a command is transmitted to the control portion 
6 over USB serial communications (process AO (a0-3) 
shown in Fig. 27 and process A3 (a3-1, a3-3) shown in 
Fig. 30). 

45 [0162] After the control portion 6 receives this com- 
mand, it divides the data stored in the internal RAM into 
data pieces and writes them into the packet control por- 
tion 14-2 in order to transmit the data over USB serial 
communications (process P0 (p0-3) shown in Fig. 19, 

50 process P2 (p2-4) shown in Fig. 21, and process P6 
(p6-1, p6-3) shown in Fig. 25). The application of the 
image source 1 -2 reconfigures the divided data pieces 
transmitted over USB serial communications, and trans- 
fers the reconfigured data to the "local clip board" under 

5S management of the GUI environment (process A3 
(a3-4) shown in Fig. 30). With the above operations, it 
is possible to transfer the image data from the image 
source 1 -1 to the image source 1 -2 by using the saving 
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[01 72] As described earlier, although USB is used as 
a means for the communications between the display 
system and image source in the above embodiments, 
the invention is not limited only thereto, but it is obvious 
that other communications means may also be used 
from the viewpoint of the characteristic features of the 
invention. Further, the embodiments use an image 
source running on the relatively highly sophisticated 
GUI environment such as Windows including a mouse 
cursor, a window layout, a cubic button and a text box. 
However, from the viewpoint of the characteristic fea- 
tures of the invention, the invention is not limited only 
thereto, but any other GUI environments may be applied 
if a user can operate the image source by selecting an 
item displayed on the display screen, such as a GUI en- 
vironment which provides a simple text operation screen 
and a simple cursor display. 

[0173] As described so far, according to the display 
system of the embodiment, a display position of the sec- 
ond position indication mark to be displayed by the sec- 
ond overlay display means is determined in accordance 
with the user information supplied via the communica- 
tions means from the user information input means, rel- 
ative to one piece of the image information selected by 
the first position indication mark displayed by the first 
overlay display means, and an operation of the second 
overlay display means is controlled in accordance with 
the determined display position. Accordingly, the display 
of images to be displayed on a conference/presentation 
display device can be controlled easily by using at least 
one image display apparatus. 

[0174] According to a display system of the embodi- 
ment, an operation of the image input means is inhibited 
in accordance with the user information supplied via the 
communications means from the user information input 
means. Accordingly, the display update/non- update of 
the image information output from each image display 
apparatus and displayed on the display device can be 
controlled by using the user information input means of 
the image display apparatus, and the display of images 
to be displayed on a conference/presentation display 
device can be controlled easily by using at least one im- 
age display apparatus. 

[01 75] According to the display system of the embod- 
iment, one piece of image information selected by the 
first position indication mark displayed by the first over- 
lay display means is read from the memory means, and 
the read one piece of image information is transmitted 
to at least one image display apparatus via the commu- 
nications means. Accordingly, one piece of image infor- 
mation selected by the first position indication mark can 
be read from the memory means storing images to be 
displayed on the display devices and can be acquired 
by at least one image display apparatus, and the display 
of images to be displayed on a conference/presentation 
display device can be controlled easily by using at least 
one image display apparatus. 

[01 76] According to the display system of the embod- 



iment, one piece of image information selected by the 
first position indication mark displayed by the first over- 
lay display means is read from the first memory means 
and temporarily stored in the second memory means, 

5 and the temporarily stored one piece of image informa- 
tion is transmitted to at least one image display appara- 
tus via the communications means. Accordingly, it is 
possible to easily transfer image information between 
image display apparatuses, and the display of images 

10 to be displayed on a conference/presentation display 
device can be controlled easily by using at least one im- 
age display apparatus. 

[0177] According to the display system of the embod- 
iment, a display position of the second position indica- 

15 tion mark to be displayed by the second overlay display 
means is input by the display position information input 
means of the display control apparatus, relative to one 
piece of the image information selected by the first po- 
sition indication mark displayed by the first overlay dis- 

20 play means, and an operation of the second overlay dis- 
play means is controlled in accordance with the input 
display position. It is therefore possible to control the 
display position of the second position indication mark 
on the display device corresponding to each image dis^ 

25 play apparatus, and the display of images to be dis- 
played on a conference/presentation display device can 
be controlled easily by using at least one image display 
apparatus. 

[01 78] A display system for receiving image data from 
30 a plurality of image sources and displaying the received 
image data on a display unit. The display system in- 
cludes: an input unit for inputting a control signal sup- 
plied from an input/output unit, a controller for controlling 
an image to be displayed on the display unit in accord- 
35 ance with the control signal input from the input unit; an 
image selection unit for selecting a desired image from 
images displayed on the display unit; and a switching 
unit for switching an image to be controlled by the con- 
troller in accordance with the image selected by the im- 
40 age selection unit. 



Claims 

45 1 . a display control apparatus having user information 
input means and connected via communications 
means to at least one image display apparatus for 
outputting image information, the display control 
apparatus comprising: 

so 

image input means for receiving at least one 
piece of image information output from said at 
least one image display apparatus; 
a display device for displaying said at least one 
55 piece of received image information; 

first overlay display means for displaying in an 
overlay manner a first position indication mark 
on said display device in order to select one 
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image display apparatus among said at least one 
image display apparatus which does not output the 
image information. 

11. A display control apparatus according to claim 9, s 
wherein said at least one image apparatus has 
memory means for storing the transmitted one 
piece of image information. 

12. A display control apparatus according to claim 9 or 10 
11 , further comprising inhibitbn means for inhibiting 

an operation of said image input means when said 
transmitting means reads the selected one piece of 
image information Irom said memory means. 

15 

13. A display control apparatus according to claim 9, 
wherein said communications means is bi-direc- 
tional serial communications means. 

14. A display control apparatus according to claim 9 or 20 
13, wherein said first overlay display means in- 
cludes display position information input means for 
inputting display position information of the first po- 
sition indication mark to be displayed on said dis- 
play device. 25 

15. A display system comprising the display control ap- 
paratus according to claim 9 and at least one image 
display apparatus connected via the communica- 
tion means to the display control apparatus and 30 
having user information input means for outputting 
image information. 

16. A display control apparatus having user information 
input means and connected via communications 35 
means to at least one image display apparatus for 
outputting image information, the display control 
apparatus comprising: 

image input means for receiving at least one 40 

piece of image information output from said at 

least one image display apparatus; 

first memory means for storing said at least one 

piece of received image information; 

second memory means for temporarily storing 45 

the image information; 

display device for displaying said at least one 
piece of received image information stored in 
said first memory means; 

first overlay display means for displaying in an so 
overlay manner a first position indication mark 
on said display device in order to select one 
piece of image information from said at least 
one piece of image information displayed on 
said display device; 55 
second overlay display means for displaying in 
an overlay manner a second position indication 
mark on said display device; and 



transmitting means for reading the one piece of 
image information selected by the first position 
indication mark displayed by said first overlay 
display means from said first memory means, 
temporarily storing the read one piece of image 
information in said second memory means, and 
transmitting the temporarily stored one piece of 
image information to said at least one image 
display apparatus via said communications 
means. 

17. A display control apparatus according to claim 16, 
wherein said at least one image display apparatus 
to which the read one piece of image information is 
transmitted by said transmitting means includes an 
image display apparatus among said at least one 
image display apparatus which does not output the 
image information. 

18. A display control apparatus according to claim 16 
wherein said at least one image apparatus has 
memory means for storing the transmitted one 
piece of image information. 

19. A display control apparatus according to claim -16 
or 18, further comprising inhibition means for inhib- 
iting an operation of said image input means when 
said transmitting means reads the selected one 
piece of image information from said first memory 
means. 

20. A display control apparatus according to claim 16, 
wherein said communications means is bi-direc- 
tional serial communications means. 

21. A display control apparatus according to claim 16 
or 18, wherein said first overlay display means in- 
cludes display position information input means for 
inputting display position information of the first po- 
sition indication mark to be displayed on said dis- 
play device. 

22. A display system comprising the display control ap- 
paratus according to claim 16 and at least one im- 
age display apparatus connected via the communi- 
cation means to the display control apparatus and 
having user information input means for outputting 
image information. 

23. A display control apparatus having user information 
input means and connected via communications 
means to at least one image display apparatus for 
outputting image information, the display control 
apparatus comprising: 

image input means for receiving at least one 
piece of image information output from said at 
least one image display apparatus; 
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plurality of image sources and displaying the re- 
ceived image data on a display unit; the display sys- 
tem comprising: 

scale varying means for varying a display scale $ 
of each image to be displayed on the display 
unit; 

a memory structured hierarchically for storing 
each image to be displayed on the display unit 
at each hierarchical level assigned a priority or- 10 
der; and 

display style setting means for setting a display 
style of each image to be displayed on the dis- 
play unit, in accordance with a scaling factor of 
the image set by said scale varying means and is 
the priority order of the image stored in said 
memory. 

37. A display system according to claim 36, further 

comprising reception means for receiving a control 20 
program Irom an external apparatus and control 
means for controlling the display style of each im- 
age to be displayed on the display unit in accord- 
ance with the control program received by said re- 
ception means, in place of said display style setting 2s 
means. 
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